{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AI Engine: GMIO Matrix Multiplication Example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The AI Engine matrix multiplication example demonstrates how to use AI Engine \n",
    "for scalar computation and use GMIO attributes for data movement. This example \n",
    "implements the standard matrix multiplication algorithm to multiply two matrices \n",
    "on the AIE cores. The user can change the matrix size and the number of cores \n",
    "utilized at compile-time. The default size is 1200x1200 (int32). The matrix \n",
    "could be further scaled to utilize the entire AIE's local data memory.\n",
    "\n",
    "Please note that this example is a proof of concept only. There could be other \n",
    "implementations that could leverage more AI Engine resources and result in \n",
    "better performance figures.\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Consider two matrices A and B, the product of the two, i.e. AxB, is a linear \n",
    "combination of the columns of A by matrix B. This means that the elements in \n",
    "a row (i) of A are multiplied with the elements in a column of B (j) and are \n",
    "summed up to give the corresponding single element in the matrix AxB at i, j. \n",
    "This means that if A is an n x m matrix and B is an m x p matrix, then the \n",
    "corresponding product AxB would have dimensions n x p. Note how the number of \n",
    "columns of A equals the number of rows in B to make the matrix multiplication \n",
    "possible.\n",
    "\n",
    "## Implementation\n",
    "\n",
    "### Data movement\n",
    "\n",
    "The application uses the GMIO attribute to make external memory-mapped connections\n",
    "to and from global memory. These connections are created between the AIE kernel and \n",
    "the logical global memory port of the hardware platform design via a Network on Chip\n",
    "(NoC). In this design, the buffer descriptors are programmed in the AIE Shim DMAs \n",
    "to initiate AIE to DDR read and write transactions from the PS program. The burst \n",
    "length of the memory-mapped transaction is 64-bit, and shim DMAs use physical \n",
    "memory addressing read/write data from global memory.\n",
    "\n",
    "<img src=\"images/data_movement.png\">\n",
    "\n",
    "\n",
    "### Data slicing\n",
    "\n",
    "To compute matrix multiplication on AIE, matrix A is sliced horizontally and \n",
    "distributed equally among all the core utilized through the AIE AXI-Stream \n",
    "network. Matrix B is transposed and feed to the first core in the design element \n",
    "by element. The first core shares the input matrix B with the other AIE cores \n",
    "through the AXI-Stream connection. As the output is in a z-order, hence a \n",
    "re-ordering of the output matrix is required.\n",
    "\n",
    "<img src=\"images/data_slicing.png\">\n",
    "\n",
    "\n",
    "## Build Flow\n",
    "\n",
    "<img src=\"images/build_flow.png\">\n",
    "\n",
    "Vitis generates aie_control_xrt.cpp, which is cross-compiled to run on the \n",
    "target. The compiled application loads the generated AIE ELFs and CDOs \n",
    "(packaged into XCLBIN) to the corresponding tile through load XCLBIN API.\n",
    "\n",
    "\n",
    "## Run-time Execution\n",
    "\n",
    "At runtime, Linux application binary calls AI Engine userspace driver,\n",
    "and runtime library, libadf_api_xrt.a. AIE userspace drivers abstract the \n",
    "kernel-space driver which handles runtime configurations along with ELF loading.\n",
    "\n",
    "<img src=\"images/runtime.png\">\n",
    "\n",
    "\n",
    "## Customizing and Rebuilding\n",
    "\n",
    "The AIE application source files are in the aie_app directory.\n",
    "\n",
    "As mentioned earlier, the user can change the number of AIE cores utilized \n",
    "for matrix multiplication. However, since the data memory immediately available\n",
    "to the core is limited, reducing the number of AIE cores reduces the maximum \n",
    "matrix size supported by the application. Within the config.h header file, \n",
    "NUM_HW_ROWS and NUM_HW_COLS macro can be set to change the number of cores \n",
    "utilized. The maximum number of AIE cores available is 400.\n",
    "\n",
    "## Directory Structure\n",
    "\n",
    "```\n",
    "plnx-aie-examples/\n",
    "├── designs\n",
    "│   └── xgemm-gmio - GMIO based AIE GeMM application\n",
    "│       ├── aie_app\n",
    "│       │   ├── kernels\n",
    "│       │   │   ├── config.h - user defined parameters\n",
    "│       │   │   ├── one_input.cc - first AIE compute kernel within an AIE row that reads data from input stream\n",
    "│       │   │   ├── one_output.cc - last AIE compute kernel within an AIE row that sends data to input stream\n",
    "│       │   │   └── two_inputs.cc - subsequent compute kernels that circulate input data and output\n",
    "│       │   ├── kernels.h - kernels declaration\n",
    "│       │   ├── Makefile\n",
    "│       │   ├── xgemm.cpp- PS main application\n",
    "│       │   └── xgemm.h - dataflow graph definition\n",
    "│       ├── hw\n",
    "│       │   ├── system.cfg - defines connections to and from PL to AIE\n",
    "│       │   └── Makefile\n",
    "│       ├── images\n",
    "│       │   ├── build_flow.png\n",
    "│       │   ├── data_movement.png\n",
    "│       │   ├── data_slicing.png\n",
    "│       │   └── runtime.png\n",
    "│       ├── Makefile\n",
    "│       ├── ps_app_hw\n",
    "│       │   └── linux\n",
    "│       │       ├── Makefile\n",
    "│       │       └── xrt.ini - XRT configuration file\n",
    "│       ├── README\n",
    "│       └── sw\n",
    "│           └── Makefile\n",
    "├── LICENSE-BINARIES\n",
    "├── LICENSE-MIT\n",
    "├── platforms\n",
    "│   ├── Makefile\n",
    "│   └── platform_create.tcl\n",
    "└── settings.sh\n",
    "```\n",
    "\n",
    "## Build Instructions\n",
    "\n",
    "Vitis and PetaLinux tools need to be installed and sourced before building the design.\n",
    "\n",
    "```\n",
    "source <Vitis_install_path>/Vitis/202X.X/settings64.sh\n",
    "source <PetaLinux_install_path>/settings.sh\n",
    "```\n",
    "\n",
    "Export the path to the VCK190 XSA and BSP \n",
    "\n",
    "```\n",
    "export BASE_XSA=\"<Base_XSA_path>\"\n",
    "export PTLNX_BSP=\"<PetaLinux_BSP_path>\"\n",
    "```\n",
    "\n",
    "Source settings.sh in a shell session.\n",
    "\n",
    "```\n",
    "source settings.sh\n",
    "```\n",
    "\n",
    "Issue Make to begin the build.\n",
    "\n",
    "```\n",
    "make\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run Demo Example\n",
    "\n",
    "Follow the PetaLinux documentation to generate boot images for your target.\n",
    "Once booted, login with the user root and password root.\n",
    "\n",
    "The AIE XCLBIN and executable are pre-installed in the /usr/bin/ directory. To run\n",
    "the demo, simply run the application \"aie-matrix-multiplication' to begin. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!aie-matrix-multiplication"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Sample Output\n",
    "\n",
    "```\n",
    "root@xilinx-vck190-2021_2:~# aie-matrix-multiplication\n",
    "Initializing ADF API...\n",
    "[INFO] AIE GMIO Matrix Multiplication\n",
    "[INFO] Matrix size(int32): 1200x1200\n",
    "[  729.529262] zocl-drm axi:zyxclmm_drm: zocl_create_client: created KDS client for pid(855), ret: 0\n",
    "[  729.538172] zocl-drm axi:zyxclmm_drm: zocl_destroy_client: client exits pid(855)\n",
    "[  729.545629] zocl-drm axi:zyxclmm_drm: zocl_create_client: created KDS client for pid(855), ret: 0\n",
    "[  729.589247] [drm] found kind 29(AIE_RESOURCES)\n",
    "[742107.428]Loading PDI from DDR\n",
    "[742107.545]Monolithic/Master Device\n",
    "[742110.856]3.399 ms: PDI initialization time\n",
    "[742114.856]+++Loading Image#: 0x0, Name: aie_image, Id: 0x1C000000\n",
    "[742120.791]---Loading Partition#: 0x0, Id: 0x0\n",
    "[742173.617] 48.585 ms for Partition#: 0x0, Size: 19032880 Bytes\n",
    "[742176.541]Subsystem PDI Load: Done\n",
    "[  729.589258] [drm] found kind 18(PDI)\n",
    "[  729.681940] [drm] FPGA Manager load DONE\n",
    "[  729.688579] [drm] Partition 1 already requested\n",
    "XAIEFAL: INFO: Resource group Avail is created.\n",
    "XAIEFAL: INFO: Resource group Static is created.\n",
    "XAIEFAL: INFO: Resource group Generic is created.\n",
    "[INFO] XCLBIN download complete\n",
    "[  729.702391] [drm] zocl_xclbin_read_axlf 9ebbbd79-4961-094c-09c8-ce28a79e87f6 ret: 0\n",
    "[INFO] AIE cores are done executing\n",
    "[INFO] Running sanity check\n",
    "[INFO] XGeMM Success!\n",
    "[  775.819920] zocl-drm axi:zyxclmm_drm: zocl_destroy_client: client exits pid(855)\n",
    "root@xilinx-vck190-2021_2:~#\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "1. AI Engine Programming Environment User Guide.\n",
    "2. Vivado User Guide - for hardware related design.\n",
    "3. Vitis User Guide - for AIE application.\n",
    "4. Versal Technical Reference Manual\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
